{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"name":"NLU_training_multi_class_text_classifier_demo.ipynb","provenance":[],"collapsed_sections":["zkufh760uvF3"]},"kernelspec":{"name":"python3","display_name":"Python 3"}},"cells":[{"cell_type":"markdown","metadata":{"id":"zkufh760uvF3"},"source":["![JohnSnowLabs](https://nlp.johnsnowlabs.com/assets/images/logo.png)\n","\n","[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/nlu/blob/master/examples/colab/Training/multi_class_text_classification/NLU_training_multi_class_text_classifier_demo.ipynb)\n","\n","\n","\n","# Training a Deep Learning Classifier with NLU \n","## ClassifierDL (Multi-class Text Classification)\n","With the [ClassifierDL model](https://nlp.johnsnowlabs.com/docs/en/annotators#classifierdl-multi-class-text-classification) from Spark NLP you can achieve State Of the Art results on any multi class text classification problem \n","\n","This notebook showcases the following features : \n","\n","- How to train the deep learning classifier\n","- How to store a pipeline to disk\n","- How to load the pipeline from disk (Enables NLU offline mode)\n","\n","\n","You can achieve these results or even better on this dataset with training data:\n","\n","<br>\n","\n","![image.png]()\n","\n","You can achieve these results or even better on this dataset with test data:\n","\n","\n","<br>\n","\n","![image.png]()\n","\n","\n"]},{"cell_type":"markdown","metadata":{"id":"dur2drhW5Rvi"},"source":["# 1. Install Java 8 and NLU"]},{"cell_type":"code","metadata":{"id":"hFGnBCHavltY","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1620188250696,"user_tz":-300,"elapsed":116328,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"a6118c32-a060-48c5-c88d-a3570a26eaa3"},"source":["!wget https://setup.johnsnowlabs.com/nlu/colab.sh -O - | bash\n","  \n","\n","import nlu"],"execution_count":null,"outputs":[{"output_type":"stream","text":["--2021-05-05 04:15:34--  https://raw.githubusercontent.com/JohnSnowLabs/nlu/master/scripts/colab_setup.sh\n","Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.111.133, 185.199.110.133, 185.199.108.133, ...\n","Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.111.133|:443... connected.\n","HTTP request sent, awaiting response... 200 OK\n","Length: 1671 (1.6K) [text/plain]\n","Saving to: ‘STDOUT’\n","\n","\r-                     0%[                    ]       0  --.-KB/s               Installing  NLU 3.0.0 with  PySpark 3.0.2 and Spark NLP 3.0.1 for Google Colab ...\n","\r-                   100%[===================>]   1.63K  --.-KB/s    in 0.001s  \n","\n","2021-05-05 04:15:35 (2.03 MB/s) - written to stdout [1671/1671]\n","\n","\u001b[K     |████████████████████████████████| 204.8MB 77kB/s \n","\u001b[K     |████████████████████████████████| 153kB 57.3MB/s \n","\u001b[K     |████████████████████████████████| 204kB 23.0MB/s \n","\u001b[K     |████████████████████████████████| 204kB 63.1MB/s \n","\u001b[?25h  Building wheel for pyspark (setup.py) ... \u001b[?25l\u001b[?25hdone\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"f4KkTfnR5Ugg"},"source":["# 2. Download news classification dataset"]},{"cell_type":"code","metadata":{"id":"OrVb5ZMvvrQD","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1620188253237,"user_tz":-300,"elapsed":118859,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"bd001162-846a-4a25-eb66-8509ad8d21f8"},"source":["! wget https://s3.amazonaws.com/auxdata.johnsnowlabs.com/public/resources/en/classifier-dl/news_Category/news_category_train.csv\n","! wget https://s3.amazonaws.com/auxdata.johnsnowlabs.com/public/resources/en/classifier-dl/news_Category/news_category_test.csv"],"execution_count":null,"outputs":[{"output_type":"stream","text":["--2021-05-05 04:17:30--  https://s3.amazonaws.com/auxdata.johnsnowlabs.com/public/resources/en/classifier-dl/news_Category/news_category_train.csv\n","Resolving s3.amazonaws.com (s3.amazonaws.com)... 52.217.81.46\n","Connecting to s3.amazonaws.com (s3.amazonaws.com)|52.217.81.46|:443... connected.\n","HTTP request sent, awaiting response... 200 OK\n","Length: 24032125 (23M) [text/csv]\n","Saving to: ‘news_category_train.csv’\n","\n","news_category_train 100%[===================>]  22.92M  17.6MB/s    in 1.3s    \n","\n","2021-05-05 04:17:31 (17.6 MB/s) - ‘news_category_train.csv’ saved [24032125/24032125]\n","\n","--2021-05-05 04:17:31--  https://s3.amazonaws.com/auxdata.johnsnowlabs.com/public/resources/en/classifier-dl/news_Category/news_category_test.csv\n","Resolving s3.amazonaws.com (s3.amazonaws.com)... 52.216.26.150\n","Connecting to s3.amazonaws.com (s3.amazonaws.com)|52.216.26.150|:443... connected.\n","HTTP request sent, awaiting response... 200 OK\n","Length: 1504408 (1.4M) [text/csv]\n","Saving to: ‘news_category_test.csv’\n","\n","news_category_test. 100%[===================>]   1.43M  2.78MB/s    in 0.5s    \n","\n","2021-05-05 04:17:32 (2.78 MB/s) - ‘news_category_test.csv’ saved [1504408/1504408]\n","\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"id":"y4xSRWIhwT28","colab":{"base_uri":"https://localhost:8080/","height":411},"executionInfo":{"status":"ok","timestamp":1620188254598,"user_tz":-300,"elapsed":120212,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"c1fab4a6-6d04-4ec2-c333-028560260aa3"},"source":["import pandas as pd\n","test_path = '/content/news_category_test.csv'\n","train_df = pd.read_csv(test_path)\n","train_df.columns=['y','text']\n","from sklearn.model_selection import train_test_split\n","\n","train_df, test_df = train_test_split(train_df, test_size=0.2)\n","train_df"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>y</th>\n","      <th>text</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>2233</th>\n","      <td>Business</td>\n","      <td>Steven Woghin, the former general counsel of C...</td>\n","    </tr>\n","    <tr>\n","      <th>5970</th>\n","      <td>Sports</td>\n","      <td>The US mens national team will look to extend ...</td>\n","    </tr>\n","    <tr>\n","      <th>3706</th>\n","      <td>Sci/Tech</td>\n","      <td>The Federal Trade Commission formally announce...</td>\n","    </tr>\n","    <tr>\n","      <th>6985</th>\n","      <td>World</td>\n","      <td>TOKYO (AP)   Japan's economy barely grew duri...</td>\n","    </tr>\n","    <tr>\n","      <th>750</th>\n","      <td>World</td>\n","      <td>Nervous Republicans are urging President Bush...</td>\n","    </tr>\n","    <tr>\n","      <th>...</th>\n","      <td>...</td>\n","      <td>...</td>\n","    </tr>\n","    <tr>\n","      <th>1537</th>\n","      <td>World</td>\n","      <td>The United States piled pressure on Sudan Wed...</td>\n","    </tr>\n","    <tr>\n","      <th>2196</th>\n","      <td>Sci/Tech</td>\n","      <td>Ask Jeeves Search Engine Gets Slim and Persona...</td>\n","    </tr>\n","    <tr>\n","      <th>553</th>\n","      <td>Sports</td>\n","      <td>Slumping Cleveland lost a three-run lead while...</td>\n","    </tr>\n","    <tr>\n","      <th>3406</th>\n","      <td>Business</td>\n","      <td>Advanced Micro Devices Inc. reported a third ...</td>\n","    </tr>\n","    <tr>\n","      <th>3888</th>\n","      <td>Sci/Tech</td>\n","      <td>A great white shark that was tagged with a da...</td>\n","    </tr>\n","  </tbody>\n","</table>\n","<p>6080 rows × 2 columns</p>\n","</div>"],"text/plain":["             y                                               text\n","2233  Business  Steven Woghin, the former general counsel of C...\n","5970    Sports  The US mens national team will look to extend ...\n","3706  Sci/Tech  The Federal Trade Commission formally announce...\n","6985     World   TOKYO (AP)   Japan's economy barely grew duri...\n","750      World   Nervous Republicans are urging President Bush...\n","...        ...                                                ...\n","1537     World   The United States piled pressure on Sudan Wed...\n","2196  Sci/Tech  Ask Jeeves Search Engine Gets Slim and Persona...\n","553     Sports  Slumping Cleveland lost a three-run lead while...\n","3406  Business   Advanced Micro Devices Inc. reported a third ...\n","3888  Sci/Tech   A great white shark that was tagged with a da...\n","\n","[6080 rows x 2 columns]"]},"metadata":{"tags":[]},"execution_count":3}]},{"cell_type":"markdown","metadata":{"id":"0296Om2C5anY"},"source":["# 3. Train Deep Learning Classifier using nlu.load('train.classifier')\n","\n","By default, the Universal Sentence Encoder Embeddings (USE) are beeing downloaded to provide embeddings for the classifier. You can use any of the 50+ other sentence Emeddings in NLU tough!\n","\n","You dataset label column should be named 'y' and the feature column with text data should be named 'text'"]},{"cell_type":"code","metadata":{"id":"3ZIPkRkWftBG","colab":{"base_uri":"https://localhost:8080/","height":1000},"executionInfo":{"status":"ok","timestamp":1620188442070,"user_tz":-300,"elapsed":307673,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"27f352ae-c42c-4685-ddab-34c897ad3a9b"},"source":["# load a trainable pipeline by specifying the train. prefix  and fit it on a datset with label and text columns\n","# Since there are no\n","fitted_pipe = nlu.load('train.classifier').fit(train_df)\n","\n","# predict with the trainable pipeline on dataset and get predictions\n","preds = fitted_pipe.predict(train_df,output_level = 'document')\n","preds"],"execution_count":null,"outputs":[{"output_type":"stream","text":["tfhub_use download started this may take some time.\n","Approximate size to download 923.7 MB\n","[OK!]\n","sentence_detector_dl download started this may take some time.\n","Approximate size to download 354.6 KB\n","[OK!]\n"],"name":"stdout"},{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>document</th>\n","      <th>text</th>\n","      <th>trained_classifier_confidence_confidence</th>\n","      <th>origin_index</th>\n","      <th>sentence</th>\n","      <th>y</th>\n","      <th>trained_classifier</th>\n","      <th>sentence_embedding_use</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>Steven Woghin, the former general counsel of C...</td>\n","      <td>Steven Woghin, the former general counsel of C...</td>\n","      <td>0.997040</td>\n","      <td>2233</td>\n","      <td>[Steven Woghin, the former general counsel of ...</td>\n","      <td>Business</td>\n","      <td>Business</td>\n","      <td>[0.035635482519865036, -0.048168957233428955, ...</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>The US mens national team will look to extend ...</td>\n","      <td>The US mens national team will look to extend ...</td>\n","      <td>1.000000</td>\n","      <td>5970</td>\n","      <td>[The US mens national team will look to extend...</td>\n","      <td>Sports</td>\n","      <td>Sports</td>\n","      <td>[0.0012551577528938651, -0.04456636682152748, ...</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>The Federal Trade Commission formally announce...</td>\n","      <td>The Federal Trade Commission formally announce...</td>\n","      <td>1.000000</td>\n","      <td>3706</td>\n","      <td>[The Federal Trade Commission formally announc...</td>\n","      <td>Sci/Tech</td>\n","      <td>Sci/Tech</td>\n","      <td>[0.010981663130223751, -0.059497587382793427, ...</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>TOKYO (AP) Japan's economy barely grew during ...</td>\n","      <td>TOKYO (AP)   Japan's economy barely grew duri...</td>\n","      <td>0.999996</td>\n","      <td>6985</td>\n","      <td>[TOKYO (AP) Japan's economy barely grew during...</td>\n","      <td>World</td>\n","      <td>Business</td>\n","      <td>[0.0031178640201687813, -0.0026406561955809593...</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>Nervous Republicans are urging President Bush ...</td>\n","      <td>Nervous Republicans are urging President Bush...</td>\n","      <td>0.999992</td>\n","      <td>750</td>\n","      <td>[Nervous Republicans are urging President Bush...</td>\n","      <td>World</td>\n","      <td>World</td>\n","      <td>[-0.018046000972390175, -0.010878069326281548,...</td>\n","    </tr>\n","    <tr>\n","      <th>...</th>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","    </tr>\n","    <tr>\n","      <th>6075</th>\n","      <td>The United States piled pressure on Sudan Wedn...</td>\n","      <td>The United States piled pressure on Sudan Wed...</td>\n","      <td>0.999965</td>\n","      <td>1537</td>\n","      <td>[The United States piled pressure on Sudan Wed...</td>\n","      <td>World</td>\n","      <td>World</td>\n","      <td>[-0.006876362022012472, 0.012149822898209095, ...</td>\n","    </tr>\n","    <tr>\n","      <th>6076</th>\n","      <td>Ask Jeeves Search Engine Gets Slim and Persona...</td>\n","      <td>Ask Jeeves Search Engine Gets Slim and Persona...</td>\n","      <td>1.000000</td>\n","      <td>2196</td>\n","      <td>[Ask Jeeves Search Engine Gets Slim and Person...</td>\n","      <td>Sci/Tech</td>\n","      <td>Sci/Tech</td>\n","      <td>[-0.0036870003677904606, -0.04579205438494682,...</td>\n","    </tr>\n","    <tr>\n","      <th>6077</th>\n","      <td>Slumping Cleveland lost a three-run lead while...</td>\n","      <td>Slumping Cleveland lost a three-run lead while...</td>\n","      <td>1.000000</td>\n","      <td>553</td>\n","      <td>[Slumping Cleveland lost a three-run lead whil...</td>\n","      <td>Sports</td>\n","      <td>Sports</td>\n","      <td>[0.03003542684018612, 0.016059285029768944, -0...</td>\n","    </tr>\n","    <tr>\n","      <th>6078</th>\n","      <td>Advanced Micro Devices Inc. reported a third q...</td>\n","      <td>Advanced Micro Devices Inc. reported a third ...</td>\n","      <td>0.980116</td>\n","      <td>3406</td>\n","      <td>[Advanced Micro Devices Inc. reported a third ...</td>\n","      <td>Business</td>\n","      <td>Business</td>\n","      <td>[0.051615066826343536, -0.005852526053786278, ...</td>\n","    </tr>\n","    <tr>\n","      <th>6079</th>\n","      <td>A great white shark that was tagged with a dat...</td>\n","      <td>A great white shark that was tagged with a da...</td>\n","      <td>0.999983</td>\n","      <td>3888</td>\n","      <td>[A great white shark that was tagged with a da...</td>\n","      <td>Sci/Tech</td>\n","      <td>Sci/Tech</td>\n","      <td>[-0.01544636394828558, -0.04387373477220535, -...</td>\n","    </tr>\n","  </tbody>\n","</table>\n","<p>6080 rows × 8 columns</p>\n","</div>"],"text/plain":["                                               document  ...                             sentence_embedding_use\n","0     Steven Woghin, the former general counsel of C...  ...  [0.035635482519865036, -0.048168957233428955, ...\n","1     The US mens national team will look to extend ...  ...  [0.0012551577528938651, -0.04456636682152748, ...\n","2     The Federal Trade Commission formally announce...  ...  [0.010981663130223751, -0.059497587382793427, ...\n","3     TOKYO (AP) Japan's economy barely grew during ...  ...  [0.0031178640201687813, -0.0026406561955809593...\n","4     Nervous Republicans are urging President Bush ...  ...  [-0.018046000972390175, -0.010878069326281548,...\n","...                                                 ...  ...                                                ...\n","6075  The United States piled pressure on Sudan Wedn...  ...  [-0.006876362022012472, 0.012149822898209095, ...\n","6076  Ask Jeeves Search Engine Gets Slim and Persona...  ...  [-0.0036870003677904606, -0.04579205438494682,...\n","6077  Slumping Cleveland lost a three-run lead while...  ...  [0.03003542684018612, 0.016059285029768944, -0...\n","6078  Advanced Micro Devices Inc. reported a third q...  ...  [0.051615066826343536, -0.005852526053786278, ...\n","6079  A great white shark that was tagged with a dat...  ...  [-0.01544636394828558, -0.04387373477220535, -...\n","\n","[6080 rows x 8 columns]"]},"metadata":{"tags":[]},"execution_count":4}]},{"cell_type":"markdown","metadata":{"id":"DL_5aY9b3jSd"},"source":["# 4. Evaluate the model"]},{"cell_type":"code","metadata":{"id":"djtoZVKBw2WU","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1620188442071,"user_tz":-300,"elapsed":307663,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"4dac293f-5bc3-41cf-dc55-c80fa608d3ff"},"source":["from sklearn.metrics import classification_report\n","print(classification_report(preds['y'], preds['classifier_dl']))\n"],"execution_count":null,"outputs":[{"output_type":"stream","text":["              precision    recall  f1-score   support\n","\n","    Business       0.85      0.80      0.82      1488\n","    Sci/Tech       0.82      0.88      0.85      1540\n","      Sports       0.95      0.97      0.96      1504\n","       World       0.90      0.87      0.89      1548\n","\n","    accuracy                           0.88      6080\n","   macro avg       0.88      0.88      0.88      6080\n","weighted avg       0.88      0.88      0.88      6080\n","\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"mhFKVN93o1ZO"},"source":["# 5. Lets try different Sentence Emebddings"]},{"cell_type":"code","metadata":{"id":"CzJd8omao0gt","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1620188442073,"user_tz":-300,"elapsed":307659,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"6596a98c-94aa-486f-e7ec-df7dd3dfff1c"},"source":["# We can use nlu.print_components(action='embed_sentence') to see every possibler sentence embedding we could use. Lets use bert!\n","nlu.print_components(action='embed_sentence')"],"execution_count":null,"outputs":[{"output_type":"stream","text":["For language <en> NLU provides the following Models : \n","nlu.load('en.embed_sentence') returns Spark NLP model tfhub_use\n","nlu.load('en.embed_sentence.use') returns Spark NLP model tfhub_use\n","nlu.load('en.embed_sentence.tfhub_use') returns Spark NLP model tfhub_use\n","nlu.load('en.embed_sentence.use.lg') returns Spark NLP model tfhub_use_lg\n","nlu.load('en.embed_sentence.tfhub_use.lg') returns Spark NLP model tfhub_use_lg\n","nlu.load('en.embed_sentence.albert') returns Spark NLP model albert_base_uncased\n","nlu.load('en.embed_sentence.electra') returns Spark NLP model sent_electra_small_uncased\n","nlu.load('en.embed_sentence.electra_small_uncased') returns Spark NLP model sent_electra_small_uncased\n","nlu.load('en.embed_sentence.electra_base_uncased') returns Spark NLP model sent_electra_base_uncased\n","nlu.load('en.embed_sentence.electra_large_uncased') returns Spark NLP model sent_electra_large_uncased\n","nlu.load('en.embed_sentence.bert') returns Spark NLP model sent_bert_base_uncased\n","nlu.load('en.embed_sentence.bert_base_uncased') returns Spark NLP model sent_bert_base_uncased\n","nlu.load('en.embed_sentence.bert_base_cased') returns Spark NLP model sent_bert_base_cased\n","nlu.load('en.embed_sentence.bert_large_uncased') returns Spark NLP model sent_bert_large_uncased\n","nlu.load('en.embed_sentence.bert_large_cased') returns Spark NLP model sent_bert_large_cased\n","nlu.load('en.embed_sentence.biobert.pubmed_base_cased') returns Spark NLP model sent_biobert_pubmed_base_cased\n","nlu.load('en.embed_sentence.biobert.pubmed_large_cased') returns Spark NLP model sent_biobert_pubmed_large_cased\n","nlu.load('en.embed_sentence.biobert.pmc_base_cased') returns Spark NLP model sent_biobert_pmc_base_cased\n","nlu.load('en.embed_sentence.biobert.pubmed_pmc_base_cased') returns Spark NLP model sent_biobert_pubmed_pmc_base_cased\n","nlu.load('en.embed_sentence.biobert.clinical_base_cased') returns Spark NLP model sent_biobert_clinical_base_cased\n","nlu.load('en.embed_sentence.biobert.discharge_base_cased') returns Spark NLP model sent_biobert_discharge_base_cased\n","nlu.load('en.embed_sentence.covidbert.large_uncased') returns Spark NLP model sent_covidbert_large_uncased\n","nlu.load('en.embed_sentence.small_bert_L2_128') returns Spark NLP model sent_small_bert_L2_128\n","nlu.load('en.embed_sentence.small_bert_L4_128') returns Spark NLP model sent_small_bert_L4_128\n","nlu.load('en.embed_sentence.small_bert_L6_128') returns Spark NLP model sent_small_bert_L6_128\n","nlu.load('en.embed_sentence.small_bert_L8_128') returns Spark NLP model sent_small_bert_L8_128\n","nlu.load('en.embed_sentence.small_bert_L10_128') returns Spark NLP model sent_small_bert_L10_128\n","nlu.load('en.embed_sentence.small_bert_L12_128') returns Spark NLP model sent_small_bert_L12_128\n","nlu.load('en.embed_sentence.small_bert_L2_256') returns Spark NLP model sent_small_bert_L2_256\n","nlu.load('en.embed_sentence.small_bert_L4_256') returns Spark NLP model sent_small_bert_L4_256\n","nlu.load('en.embed_sentence.small_bert_L6_256') returns Spark NLP model sent_small_bert_L6_256\n","nlu.load('en.embed_sentence.small_bert_L8_256') returns Spark NLP model sent_small_bert_L8_256\n","nlu.load('en.embed_sentence.small_bert_L10_256') returns Spark NLP model sent_small_bert_L10_256\n","nlu.load('en.embed_sentence.small_bert_L12_256') returns Spark NLP model sent_small_bert_L12_256\n","nlu.load('en.embed_sentence.small_bert_L2_512') returns Spark NLP model sent_small_bert_L2_512\n","nlu.load('en.embed_sentence.small_bert_L4_512') returns Spark NLP model sent_small_bert_L4_512\n","nlu.load('en.embed_sentence.small_bert_L6_512') returns Spark NLP model sent_small_bert_L6_512\n","nlu.load('en.embed_sentence.small_bert_L8_512') returns Spark NLP model sent_small_bert_L8_512\n","nlu.load('en.embed_sentence.small_bert_L10_512') returns Spark NLP model sent_small_bert_L10_512\n","nlu.load('en.embed_sentence.small_bert_L12_512') returns Spark NLP model sent_small_bert_L12_512\n","nlu.load('en.embed_sentence.small_bert_L2_768') returns Spark NLP model sent_small_bert_L2_768\n","nlu.load('en.embed_sentence.small_bert_L4_768') returns Spark NLP model sent_small_bert_L4_768\n","nlu.load('en.embed_sentence.small_bert_L6_768') returns Spark NLP model sent_small_bert_L6_768\n","nlu.load('en.embed_sentence.small_bert_L8_768') returns Spark NLP model sent_small_bert_L8_768\n","nlu.load('en.embed_sentence.small_bert_L10_768') returns Spark NLP model sent_small_bert_L10_768\n","nlu.load('en.embed_sentence.small_bert_L12_768') returns Spark NLP model sent_small_bert_L12_768\n","For language <fi> NLU provides the following Models : \n","nlu.load('fi.embed_sentence') returns Spark NLP model sent_bert_finnish_cased\n","nlu.load('fi.embed_sentence.bert.cased') returns Spark NLP model sent_bert_finnish_cased\n","nlu.load('fi.embed_sentence.bert.uncased') returns Spark NLP model sent_bert_finnish_uncased\n","For language <xx> NLU provides the following Models : \n","nlu.load('xx.embed_sentence') returns Spark NLP model sent_bert_multi_cased\n","nlu.load('xx.embed_sentence.bert') returns Spark NLP model sent_bert_multi_cased\n","nlu.load('xx.embed_sentence.bert.cased') returns Spark NLP model sent_bert_multi_cased\n","nlu.load('xx.embed_sentence.labse') returns Spark NLP model labse\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"id":"ABHLgirmG1n9","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1620191283243,"user_tz":-300,"elapsed":3148824,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"50149867-b85c-4f1a-ff9d-d0af0077524d"},"source":["# Load pipe with bert embeds\n","# using large embeddings can take a few hours..\n","# fitted_pipe = nlu.load('en.embed_sentence.bert_large_uncased train.classifier').fit(train_df)\n","fitted_pipe = nlu.load('en.embed_sentence.small_bert_L12_768 train.classifier').fit(train_df)\n","\n","\n","# predict with the trained pipeline on dataset and get predictions\n","preds = fitted_pipe.predict(train_df,output_level='document')\n","print(classification_report(preds['y'], preds['classifier_dl']))\n"],"execution_count":null,"outputs":[{"output_type":"stream","text":["sent_small_bert_L12_768 download started this may take some time.\n","Approximate size to download 392.9 MB\n","[OK!]\n","sentence_detector_dl download started this may take some time.\n","Approximate size to download 354.6 KB\n","[OK!]\n","              precision    recall  f1-score   support\n","\n","    Business       0.84      0.86      0.85      1488\n","    Sci/Tech       0.87      0.85      0.86      1540\n","      Sports       0.94      0.98      0.96      1504\n","       World       0.92      0.88      0.90      1548\n","\n","    accuracy                           0.89      6080\n","   macro avg       0.89      0.89      0.89      6080\n","weighted avg       0.89      0.89      0.89      6080\n","\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"_1jxw3GnVGlI"},"source":["# 5 evaluate on Test Data"]},{"cell_type":"code","metadata":{"id":"Fxx4yNkNVGFl","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1620191537907,"user_tz":-300,"elapsed":236219,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"645bfa57-4451-4878-dd85-b2f774a78092"},"source":["preds = fitted_pipe.predict(test_df,output_level='document')\n","\n","#sentence detector that is part of the pipe generates sone NaNs. lets drop them first\n","preds.dropna(inplace=True)\n","print(classification_report(preds['y'], preds['classifier_dl']))"],"execution_count":null,"outputs":[{"output_type":"stream","text":["              precision    recall  f1-score   support\n","\n","    Business       0.86      0.86      0.86       412\n","    Sci/Tech       0.87      0.86      0.87       360\n","      Sports       0.95      0.97      0.96       396\n","       World       0.90      0.89      0.90       352\n","\n","    accuracy                           0.90      1520\n","   macro avg       0.90      0.90      0.90      1520\n","weighted avg       0.90      0.90      0.90      1520\n","\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"2BB-NwZUoHSe"},"source":["# 6. Lets save the model"]},{"cell_type":"code","metadata":{"id":"eLex095goHwm","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1620191735880,"user_tz":-300,"elapsed":434174,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"bf2fab86-94d6-49a1-f2ad-783bc56ca450"},"source":["stored_model_path = './models/classifier_dl_trained' \n","fitted_pipe.save(stored_model_path)"],"execution_count":null,"outputs":[{"output_type":"stream","text":["Stored model in ./models/classifier_dl_trained\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"e_b2DPd4rCiU"},"source":["# 7. Lets load the model from HDD.\n","This makes Offlien NLU usage possible!   \n","You need to call nlu.load(path=path_to_the_pipe) to load a model/pipeline from disk."]},{"cell_type":"code","metadata":{"id":"SO4uz45MoRgp","colab":{"base_uri":"https://localhost:8080/","height":95},"executionInfo":{"status":"ok","timestamp":1620191750307,"user_tz":-300,"elapsed":448594,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"86286c2a-4eb4-46e0-ddf4-7088b1c2375f"},"source":["hdd_pipe = nlu.load(path=stored_model_path)\n","\n","preds = hdd_pipe.predict('Tesla plans to invest 10M into the ML sector')\n","preds"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>document</th>\n","      <th>from_disk</th>\n","      <th>from_disk_confidence_confidence</th>\n","      <th>text</th>\n","      <th>origin_index</th>\n","      <th>sentence_embedding_from_disk</th>\n","      <th>sentence</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>Tesla plans to invest 10M into the ML sector</td>\n","      <td>[Business]</td>\n","      <td>[0.6325806]</td>\n","      <td>Tesla plans to invest 10M into the ML sector</td>\n","      <td>8589934592</td>\n","      <td>[[0.15737193822860718, 0.2598555386066437, 0.8...</td>\n","      <td>[Tesla plans to invest 10M into the ML sector]</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["                                       document  ...                                        sentence\n","0  Tesla plans to invest 10M into the ML sector  ...  [Tesla plans to invest 10M into the ML sector]\n","\n","[1 rows x 7 columns]"]},"metadata":{"tags":[]},"execution_count":11}]},{"cell_type":"code","metadata":{"id":"e0CVlkk9v6Qi","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1620191750314,"user_tz":-300,"elapsed":448025,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"b5f147f2-0da0-4d4d-d53e-8d10a9be5c7e"},"source":["hdd_pipe.print_info()"],"execution_count":null,"outputs":[{"output_type":"stream","text":["The following parameters are configurable for this NLU pipeline (You can copy paste the examples) :\n",">>> pipe['document_assembler'] has settable params:\n","pipe['document_assembler'].setCleanupMode('shrink')                                                    | Info: possible values: disabled, inplace, inplace_full, shrink, shrink_full, each, each_full, delete_full | Currently set to : shrink\n",">>> pipe['sentence_detector@SentenceDetectorDLModel_c83c27f46b97'] has settable params:\n","pipe['sentence_detector@SentenceDetectorDLModel_c83c27f46b97'].setExplodeSentences(False)              | Info: whether to explode each sentence into a different row, for better parallelization. Defaults to false. | Currently set to : False\n","pipe['sentence_detector@SentenceDetectorDLModel_c83c27f46b97'].setStorageRef('SentenceDetectorDLModel_c83c27f46b97')  | Info: storage unique identifier | Currently set to : SentenceDetectorDLModel_c83c27f46b97\n","pipe['sentence_detector@SentenceDetectorDLModel_c83c27f46b97'].setEncoder(com.johnsnowlabs.nlp.annotators.sentence_detector_dl.SentenceDetectorDLEncoder@6934a3e6)  | Info: Data encoder | Currently set to : com.johnsnowlabs.nlp.annotators.sentence_detector_dl.SentenceDetectorDLEncoder@6934a3e6\n","pipe['sentence_detector@SentenceDetectorDLModel_c83c27f46b97'].setImpossiblePenultimates(['Bros', 'No', 'al', 'vs', 'etc', 'Fig', 'Dr', 'Prof', 'PhD', 'MD', 'Co', 'Corp', 'Inc', 'bros', 'VS', 'Vs', 'ETC', 'fig', 'dr', 'prof', 'PHD', 'phd', 'md', 'co', 'corp', 'inc', 'Jan', 'Feb', 'Mar', 'Apr', 'Jul', 'Aug', 'Sep', 'Sept', 'Oct', 'Nov', 'Dec', 'St', 'st', 'AM', 'PM', 'am', 'pm', 'e.g', 'f.e', 'i.e'])  | Info: Impossible penultimates | Currently set to : ['Bros', 'No', 'al', 'vs', 'etc', 'Fig', 'Dr', 'Prof', 'PhD', 'MD', 'Co', 'Corp', 'Inc', 'bros', 'VS', 'Vs', 'ETC', 'fig', 'dr', 'prof', 'PHD', 'phd', 'md', 'co', 'corp', 'inc', 'Jan', 'Feb', 'Mar', 'Apr', 'Jul', 'Aug', 'Sep', 'Sept', 'Oct', 'Nov', 'Dec', 'St', 'st', 'AM', 'PM', 'am', 'pm', 'e.g', 'f.e', 'i.e']\n","pipe['sentence_detector@SentenceDetectorDLModel_c83c27f46b97'].setModelArchitecture('cnn')             | Info: Model architecture (CNN) | Currently set to : cnn\n",">>> pipe['bert_sentence@sent_small_bert_L12_768'] has settable params:\n","pipe['bert_sentence@sent_small_bert_L12_768'].setBatchSize(8)                                          | Info: Size of every batch | Currently set to : 8\n","pipe['bert_sentence@sent_small_bert_L12_768'].setCaseSensitive(False)                                  | Info: whether to ignore case in tokens for embeddings matching | Currently set to : False\n","pipe['bert_sentence@sent_small_bert_L12_768'].setDimension(768)                                        | Info: Number of embedding dimensions | Currently set to : 768\n","pipe['bert_sentence@sent_small_bert_L12_768'].setMaxSentenceLength(128)                                | Info: Max sentence length to process | Currently set to : 128\n","pipe['bert_sentence@sent_small_bert_L12_768'].setIsLong(False)                                         | Info: Use Long type instead of Int type for inputs buffer - Some Bert models require Long instead of Int. | Currently set to : False\n","pipe['bert_sentence@sent_small_bert_L12_768'].setStorageRef('sent_small_bert_L12_768')                 | Info: unique reference name for identification | Currently set to : sent_small_bert_L12_768\n",">>> pipe['classifier_dl@sent_small_bert_L12_768'] has settable params:\n","pipe['classifier_dl@sent_small_bert_L12_768'].setClasses(['World', 'Sci/Tech', 'Sports', 'Business'])  | Info: get the tags used to trained this ClassifierDLModel | Currently set to : ['World', 'Sci/Tech', 'Sports', 'Business']\n","pipe['classifier_dl@sent_small_bert_L12_768'].setStorageRef('sent_small_bert_L12_768')                 | Info: unique reference name for identification | Currently set to : sent_small_bert_L12_768\n"],"name":"stdout"}]}]}